fix(typegen): accept deserialized json/jsonb values in generated types for Python - #1129
Closed
h-zahar wants to merge 1 commit into
Closed
fix(typegen): accept deserialized json/jsonb values in generated types for Python#1129h-zahar wants to merge 1 commit into
h-zahar wants to merge 1 commit into
Conversation
Contributor
|
Thank you for the contribution! postgres-meta's type generation is moving to the shared |
spydon
added a commit
to supabase/sdk
that referenced
this pull request
Sep 1, 2026
…thon pydantic's Json[Any] validates a JSON string and deserializes it, but PostgREST returns json and jsonb columns already deserialized, so every generated model with a JSON column failed model_validate. JsonValue is pydantic's type for an already parsed JSON value. Ported from supabase/postgres-meta#1129
spydon
added a commit
to supabase/sdk
that referenced
this pull request
Sep 1, 2026
…columns, composite nullability (#124) ## Summary Ports the worthwhile Python generator fixes from postgres-meta's open template PRs into this package (the templates are being deleted in favor of this package in supabase/postgres-meta#1084, so open fixes there are triaged and re-landed here). Four fixes, one commit each: 1. **Identifier escaping** (from supabase/postgres-meta#1082): enum `Literal` labels and `Field(alias=...)` values were interpolated unescaped, so a quote, backslash, or newline in a database name broke the generated module. A shared `escapePythonString` helper (JSON escaping, a strict subset of Python's) now covers all three interpolation sites. 2. **Python 3.9/3.10 support** (from supabase/postgres-meta#1094): `NotRequired` (3.11+) and `TypeAlias` (3.10+) now import from `typing_extensions`, which is always installed as a required dependency of pydantic. 3. **Deserialized json/jsonb** (from supabase/postgres-meta#1129): `json`/`jsonb` map to pydantic's `JsonValue` instead of `Json[Any]`. PostgREST returns these columns already deserialized, while `Json[Any]` validates a JSON *string* and parses it, so every generated model with a JSON column failed `model_validate` (supabase/supabase-py#1597). 4. **Composite type nullability** (the Python side of supabase/postgres-meta#1063, reimplemented): composite type attributes cannot carry NOT NULL constraints in Postgres, so their fields now emit `Optional[...]`. The origin PR's Python hunks were dead code (an unused `PythonDomain` class and a type-map entry for a name Postgres never emits), so the actual fix was implemented instead of ported. ## Triage of origin PRs | postgres-meta PR | Verdict | Reasoning | |---|---|---| | #1082 | Ported | Real invalid-syntax bug, correct approach. | | #1094 | Ported | Import failure on Python 3.9/3.10, independently verified by community comments on the PR. | | #1129 | Ported | Every JSON column failed validation at runtime; `JsonValue` is pydantic's native type for a parsed JSON value. | | #1063 (python part) | Reimplemented | Real bug, but the PR's Python changes did not actually fix it (dead code); the underlying fix is one line in `typeToClass`. | | #1072 (`frozen=True`) | Skipped | Author-labeled feature and an opinionated behavior change that breaks consumers who mutate row models; belongs behind a generator option if wanted. | | #808 | Skipped | 2023 draft fully superseded by the maintainer-authored template this package ports. | ## Validation - Unit tests per fix (pathological enum labels and aliases, import block assertions, json/jsonb mapping, composite `Optional` fields). - Parity golden regenerated (39 lines): the `typing_extensions` import split, 16 `Json[Any]` to `JsonValue` occurrences, and two composite attributes gaining `Optional[...]`; reviewed line by line and the golden gate was verified to actually trip on corruption. - The regenerated golden imports cleanly under pydantic, passes `mypy`, and runtime checks confirm deserialized JSON and `None` composite fields now validate. - `check-types`, `format-and-lint`, `knip`, `build`, `test` (97 pass, includes Docker-backed introspection and parity) all green. - Note: the nightly parity job against real postgres-meta will show this intentional drift until postgres-meta consumes a release containing it (supabase/postgres-meta#1084 replaces the templates with this package, closing the gap).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The issue was reported in supabase/supabase-py#1597.
Currently, the Python generator maps
jsonandjsonbcolumns to pydantic'sJson[Any], which expects astrcontaining JSON and deserializes it. PostgREST returns those columns already deserialized, so every generated model with a JSON column raises onmodel_validate: JSON input should be string, bytes or bytearray.With this PR, both column types map to
pydantic.JsonValue, which denotes an already-deserialized JSON value.The issue is resolved using pydantic's native type without defining a custom type.